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ABSTRACT 


Objectives:  The  purpose  of  this  study  was  to  evaluate  oculometrics  as  a  detector  of  fatigue  in  a 
Air  Force  relevant  environments  using  one  night  of  sleep  deprivation.  Using  the  eye  metrics  of 
total  eye  closure  duration  and  approximate  entropy,  the  relationship  between  these  eye  metrics 
and  the  onset  of  fatigue-induced  perfonnance  decrements  was  investigated.  Background: 
Perhaps,  the  most  damaging  effects  to  the  successful  outcome  of  operational  military  missions 
are  those  attributed  to  sleep  deprivation-induced  fatigue.  Consequently,  there  is  increasing 
interest  in  the  development  of  reliable  monitoring  devices  that  can  easily  assess  when  an  operator 
or  soldier  may  be  overly  fatigued  and  operating  at  a  dangerous  performance  level.  Method:  Ten 
civilian  subjects  volunteered  to  participate  in  this  study.  Each  was  trained  on  three  performance 
tasks:  target  identification,  unmanned  aerial  vehicle  (UAV)  landing,  and  the  psychomotor 
vigilance  task  (PVT).  Experimental  testing  of  the  three  tasks  began  after  14  hours  of  sleep 
deprivation,  and  continued  every  two  hours  until  28  hours  of  sleep  deprivation  was  reached. 
Results:  Data  analyses  showed  statistically  significant  decrements  in  performance  as  the  level  of 
sleep  deprivation  increased,  for  both  the  PVT  and  the  target  identification  task.  These 
perfonnance  declines  correlated  with  increases  in  proportion  of  eye  closure  and  declines  in 
approximate  entropy  of  pupil  position.  Conclusion:  The  results  provide  evidence  that  eye 
metrics  can  be  used  to  detect  the  onset  of  fatigue,  potentially  in  advance  of  significant  changes  in 
operator  perfonnance,  suggesting  a  way  to  predict  fatigue-induced  declines  in  perfonnance 
before  they  manifest.  Application:  Operators  in  both  manned  and  unmanned  vehicle 
environments,  as  well  as  other  military  or  commercial  operator  environments,  could  benefit 
greatly  from  an  alertness  monitoring  device  for  assessing  fatigue  status  and  task  performance. 


INTRODUCTION 


The  operational  military  environment  presents  a  variety  of  environmental  and  physiological 
stressors  such  as  vibration,  heat,  high  acceleration,  and  fatigue  that  can  significantly  influence  the 
effectiveness  and  task  perfonnance  of  the  individual.  Perhaps  the  most  damaging  effects  to  the 
successful  outcome  of  the  mission  are  those  attributed  to  sleep  deprivation-induced  fatigue.  Although 
considerable  research  has  been  conducted  to  assuage  the  effects  of  pilot  fatigue,  it  remains  a 
significant  concern  in  aviation  operations.  NASA"s  Aviation  Safety  Reporting  System  (ASRS) 
routinely  receives  reports  from  pilots  blaming  fatigue,  sleep  loss,  and  sleepiness  in  the  cockpit  for 
operational  errors  such  as  altitude  and  course  deviations,  fuel  miscalculations,  landings  without  proper 
clearances,  and  landings  on  incorrect  runways  (Caldwell,  Caldwell,  and  Schmidt,  2008;  Rosekind  et 
al.,  1994).  Aviator  fatigue  is  associated  with  degradations  in  response  accuracy  and  speed,  the 
unconscious  acceptance  of  lower  standards  of  perfonnance,  impairments  in  the  capacity  to  integrate 
infonnation,  and  nanowing  of  attention  that  can  lead  to  forgetting  or  ignoring  important  aspects  of 
flight  tasks  (Peny,  1974).  As  sleepiness  levels  increase,  performance  becomes  less  consistent  and 
vigilance  deteriorates  (Dinges,  1990). 

There  is  a  need  to  accurately  detennine  when  pilots  become  fatigued  to  dangerous  levels,  but 
progress  in  the  area  of  “fitness  for  duty”  testing  and  “real-time  monitoring”  of  operator  performance 
has  been  slow  (Institute  of  Medicine,  2004).  Oculometric-based  techniques  appear  as  a  promising 
measure  for  fatigue.  Stern  (1999)  has  shown  that  oculomotor  measures  are  useful  for  detecting  lapses 
in  attention,  and  several  other  investigators  (Dinges  et  al.,  1998  and  Russo  et  al.,  1999)  have 
established  the  sensitivity  of  the  oculomotor  control  system  to  detect  fatigue,  boredom,  and  lapses  in 
attention.  Russo  et  al.  (2003)  reported  that  saccadic  velocity  is  particularly  sensitive  to  an  increase  in 
sleepiness  in  response  to  prolonged  periods  of  partial  sleep  deprivation,  and  Russo  et  al.  (1999)  found 
that  both  decreases  in  saccadic  velocity  and  increases  in  pupil  constriction  latency  correlated  with  an 
increase  in  the  rate  of  crashes  under  simulated  driving  conditions  during  periods  of  sleep  deprivation. 
Long  duration  eye  closures  during  eye  blinks  also  have  been  found  to  provide  a  good  indication  of 
reduced  alertness  (Stern,  1980),  and  others  have  demonstrated  that  an  integrated  measure  of  the  degree 
of  eye  closure  over  a  specific  interval  of  time  offers  infonnation  about  the  level  of  operator  sleepiness 
(Wierwille,  1999).  Specifically,  Dinges  et  al.  (1998)  found  a  high  degree  of  coherence  between  slow 
eye  closures  (measured  via  PERCLOS  [Percent  Eye  Closure])  and  performance  lapses  on  a  test  of 
sustained  attention,  and  Mallis  et  al.  (2000)  found  that  PERCLOS  feedback  from  the  device  improved 
alertness  and  driving  performance,  especially  when  drivers  were  drowsy  at  night.  In  part,  this  is  why 
the  Federal  Highway  Administration  and  the  National  Highway  Traffic  Safety  Administration  consider 
PERCLOS  to  be  among  the  most  promising  known  real-time  measures  of  alertness  for  in-vehicle 
drowsiness  detection  (Dinges  and  Grace,  1998). 
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METHOD 


The  goal  of  this  experiment  was  to  detennine  if  eye  metrics  could  detect  fatigue  before 
degradations  in  performance. 


Participants 

An  Institutional  Review  Board  (IRB)  approved  experimental  protocol  was  used  in  this  study. 
A  total  of  10  volunteer  subjects  (9  male,  1  female)  were  admitted  to  the  study  after  providing 
informed  consent  to  participate.  They  ranged  in  age  from  18  to  42  years.  Female  subjects  that  may 
have  been  pregnant  (determined  by  a  urine  pregnancy  test)  were  not  pennitted  to  participate. 


Apparatus 

EC6  Eye-Tracker:  The  study  required  each  subject  to  wear  the  EyeCom  (Reno,  NV)  eye- 
tracker  (EC6)  during  testing.  The  device  consists  of  two  infrared  (IR)-sensitive  cameras  and  a  linear 
array  of  IR-illuminating  light  emitting  diodes  (LEDs)  mounted  on  a  set  of  eyeglass  frames.  The 
cameras  are  angled  upward  toward  the  eyes  and  extract  real-time  pupil  diameter,  eye-lid  movement, 
and  eye -ball  movement.  The  software  records  a  variety  of  measurements  including  pupil  position,  eye 
closure,  size,  etc.  which  can  then  be  used  to  calculate  eye -blink  duration  (EBD),  eye -blink  frequency 
(EBF),  eye-blink  velocity  (EBV),  percentage  of  time  the  eyes  are  closed  (PERCLOS),  saccadic  eye 
movement  velocity,  and  pupil  response  latency  to  light  flashes.  Only  the  left  eye  was  monitored  for 
this  experiment  as  left  and  right  eye  comparisons  were  not  necessary  for  the  planned  analyses. 

Actigraph  Monitor:  Two  days  prior  to  data  collection,  each  subject  donned  a  wrist  activity 
monitor  (WAM;  Ambulatory  Monitoring,  Inc.).  The  WAM  is  a  non-invasive  small  electronic  device 
that  can  be  worn  on  the  wrist  like  a  wristwatch.  It  records  limb  and  body  movements  to  detennine 
when  a  subject  is  active  and  when  they  are  asleep.  It  was  used  to  ensure  the  subjects  received  at  least  7 
hours  of  sleep  per  night  for  the  two  nights  prior  to  data  collection. 

Simulation  Facilities:  Two  simulation  facilities  were  used  in  this  study.  The  first  was  the 
Predator  RQ-1  Ground  Control  Station  (GCS).  This  simulation  apparatus  utilizes  two  19-inch  cathode 
ray  tube  (CRT)  monitors  to  display  a  map  view  and  a  nose-camera  view  for  the  simulation.  The 
positions  of  the  monitors  were  the  same  as  that  of  the  operational  Predator  GCS  (top  monitor  at  15° 
downward  angle  44. 5 -inches  above  the  table,  bottom  monitor  perpendicular  with  the  table  at  eye 
height).  An  overhead  map  view  was  displayed  on  the  top  monitor,  and  a  camera  view  (from  the  nose 
of  the  simulated  aircraft)  was  displayed  on  the  bottom  monitor,  as  is  the  case  for  the  operational  GCS. 
The  seat  utilized  was  a  traditional  office  chair  with  an  adjustable  seat  height  (similar  to  the  chair  used 
by  Predator  pilots).  A  replicated  Predator  GCS  flight  stick  and  throttle  developed  by  High  Rev 
Simulators  (Lancaster,  CA)  served  as  the  flight  control  interface.  The  landing  task  included  one  pre¬ 
determined  flight  path  the  subject  was  required  to  follow  each  trial.  The  flight  software  was  a 
modified  version  of  X-Plane  Flight  Simulator,  version  7.0  (Laminar  Research,  Columbia,  SC),  run  on 
a  Gateway  Pentium  IV  desktop  PC  (Gateway  Inc.,  Irvine,  CA).  An  experienced  pilot  consultant 
ensured  that  the  flight  dynamics  and  displays  were  similar  to  those  employed  by  the  Predator 
Unmanned  Aerial  Vehicle  UAV.  X-plane  provided  output  data  that  were  recorded  by  a  custom 
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program  written  in  Microsoft  Visual  C++  (Microsoft  Corp,  Seattle,  WA).  This  output  data  included 
glideslope  path,  altitude,  airspeed,  descent  rate,  heading,  and  position.  Additional  software  was  coded 
in  OpenGL  (Silicon  Graphics,  Inc,  Mountain  View,  CA)  and  interfaced  with  the  X-plane  program  to 
display  the  map  and  heads-up  display  (HUD)  symbology  overlay. 

The  second  simulator  was  the  Synthesized  Immersion  Research  Environment  (SIRE),  which  is 
a  state-of-the-art  virtual  environment  research  laboratory.  This  facility  contains  a  40-foot  diameter 
dome  that  serves  as  a  high-resolution,  large  field-  of-view  (70  degrees  vertical  by  150  degrees 
horizontal)  interactive  visual  display.  A  simulated  UH-60  helicopter  cockpit  developed  by  Protobox, 
Inc.  (Dayton,  OH)  resides  in  the  center  of  the  dome  display.  The  cockpit  includes  both  the  left  and 
right  seats,  instrument  cluster,  cyclic,  collective,  and  pedal  controls,  and  a  simulated  UH-60 
surrounding  structure.  This  station  also  included  an  electro-hydraulic  control  loader  system  to  raise 
the  simulated  UH-60  cockpit  to  the  desired  height.  Subjects  participating  in  the  study  were  required  to 
perform  a  target  identification  task  within  a  simulated  flight  environment.  Although  the  simulation  is 
capable  of  being  flown  manually  by  a  human  operator,  all  flights  used  in  this  experiment  were 
computer  controlled  due  to  the  fact  the  subjects  were  simply  required  to  visually  search  for  targets. 

Psychomotor  Vigilance  Task  (PVT):  The  PVT- 192  psychomotor  vigilance  task  (Ambulatory 
Monitoring,  Inc.;  Ardsley,  NY)  is  a  handheld,  computerized  test  presentation  and  data  capture  system 
that  records  simple  reaction  times  on  a  visual  detection  task,  known  to  be  sensitive  to  sleep  loss 
(Dinges  et  al.,  1997).  The  visual  stimulus  is  presented  on  a  small  liquid  crystal  display  (LCD)  and 
subject  responses  are  secured  using  two  separate  buttons.  The  PVT  requires  sustained  attention  and 
discrete  motor  responses.  The  8"  x  4.5"  x  2.4"  portable,  battery-operated  device  visually  displays 
numbers  counted  up  by  milliseconds  in  a  window. 

Grass  Telefactor  Electroencephalograph:  Each  subject  was  instrumented  with 
electroencephalograph  (EEG)  sensors  to  monitor  brain  activity  during  testing.  The  sensors  were 
attached  to  the  scalp  with  medical  grade  adhesive  tape.  The  Federal  Drug  Administration  (FDA)- 
approved  Grass  Telefactor  (Astro-Med,  Inc.;  West  Warwick,  RI)  EEG  system  includes  an  amplifier 
and  a  computer  to  record  and  analyze  the  data.  The  amplifier  can  accept  up  to  32  sensors  and  samples 
data  up  to  400  hertz  (Hz).  Mindline  channels  Fpi,  C3,  C4,  Fz,  Cz,  Pz,  and  Oz  from  the  standard  10-20 
sensor  placements  system  were  selected  for  the  montage  used  in  this  experiment  and  were  referenced 
to  electrode  sites  Ai  and  A2  (located  behind  the  ears)  during  recording.  The  time  constant  for  the 
EEG  channels  was  0.3  second,  and  the  high-pass  filter  was  set  to  70  Hz. 


Stimuli 

Subjects  were  required  to  complete  three  separate  perfonnance  tasks  for  this  study:  target 
identification,  UAV  landing,  and  PVT.  The  target  identification  task  was  performed  in  the  right  seat 
of  the  UH-60  helicopter  simulator  within  the  SIRE  facility.  The  simulated  aircraft  was  computer 
controlled  and  flew  straight  and  level  at  5000  ft.  Weather  conditions  were  set  to  “clear”  and  time  of 
day  was  12:00  pm  to  optimize  visual  conditions  and  allow  subjects  to  readily  see  each  target  aircraft. 

During  each  trial,  three  target  aircraft  were  simultaneously  presented  at  a  simulated  distance  of 
200  ft  from  the  subject' 's  aircraft.  The  aircraft  types  for  the  three  targets  were  randomized  with  the 
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condition  that  at  least  two  of  the  aircraft  must  be  different.  A  total  of  fifty  trials  were  administered  in 
each  of  seven  sessions.  Each  target' 's  elevation  and  azimuth  were  randomly  generated  within  the 
limitations  of  the  SIRE  screen  field  of  view.  Elevation  was  limited  to  ±20°  whereas  azimuth  was 
limited  to  ±50°.  Three  types  of  target  aircraft  were  displayed  including  enemy  (Su-37),  friendly  (F- 
22),  and  unknown  (F-16).  These  aircraft  were  chosen  due  to  their  similarity  in  size  ensuring  equal 
salience  on  the  screen.  Subjects  were  required  to  visually  acquire  each  target  by  fixating  a  laser  point 
beam  (laser  pointer  was  attached  to  the  top  of  their  helmet)  in  the  center  of  the  target  (+  2  degrees  of 
visual  arc)  for  4  seconds.  Head  tracking  was  accomplished  using  a  Polhamus  head  tracker 
(Colchester,  VT).  Subjects  were  instructed  to  acquire  the  targets  in  the  order  of  all  enemies,  all 
unknown,  and  then  all  friendlies. 

The  simulated  UAV  landing  task  was  generated  in  the  X-plane  software  (Laminar  Research, 
Columbia,  SC)  environment.  The  subjects  flew  a  simulated  MQ-1  Predator  that  was  positioned  at  800 
ft  of  altitude  near  the  main  runway  located  at  the  Indian  Springs  Airport  (Creech  Air  Force  Base)  in 
Nevada.  Three  waypoints  were  situated  in  the  terrain  database.  The  first  waypoint  was  at  800  ft  of 
altitude,  the  second  was  located  at  600  ft,  and  the  third  was  set  to  360  ft.  Two  of  these  waypoints  were 
indicated  on  the  map  with  red  dots  and  the  third  was  positioned  0.9  mile  out  from  the  runway  (in  line 
with  the  runway).  The  standard  MQ-1  Predator  symbology  was  overlaid  on  the  nose  camera  monitor. 
The  task  required  each  subject  to  fly  through  each  waypoint  at  the  specified  altitude  and  land 
successfully  on  the  runway.  Successful  landings  were  quantified  by  using  three  performance 
parameters.  Subjects  were  instructed  to  achieve  a  glideslope  of  less  than  20  feet,  a  touchdown  vertical 
velocity  less  than  220  feet  per  second,  and  a  touchdown  airspeed  of  under  70  knots.  To  increase  the 
difficulty,  visibility  was  set  to  0.8  nautical  miles  (NM)  and  the  time  of  day  was  set  to  midnight. 

The  PVT  task  consisted  digital  number  presented  on  a  small  LCD  screen  that  started  at  zero 
and  counted  up  sequentially  by  milliseconds  (up  to  60,000  ms,  or  1  min)  until  the  response  button  was 
pressed.  When  the  subjects  detected  the  start  of  the  task,  they  pressed  a  microswitch  that  recorded  the 
reaction  time  to  the  stimulus  and  cleared  the  screen  for  the  next  trial.  If  the  subjects  failed  to  detect 
task,  a  time-out  was  recorded.  The  interstimulus  interval  varied  randomly  from  2  to  12  seconds.  The 
data  were  stored  on  computer  and  processed  by  custom  software  for  subsequent  analysis. 


Procedure 

Subjects  were  provided  training  on  all  three  performance  tasks  over  a  period  of  at  least  three 
separate  days.  Training  was  completed  once  their  task  performance  remained  within  10%  of  their 
previous  training  session.  Extra  training  was  administered  when  necessary.  Data  collection  was  held 
the  next  day  after  they  were  trained.  Refresher  training  was  only  given  if  they  missed  the  data 
collection  appointment.  Two  days  prior  to  experimental  trials,  participants  were  given  an  activity  wrist 
monitor  and  instructed  that  their  daily  schedules  should  include  a  minimum  of  seven  hours  sleep  per 
night. 


On  the  testing  day,  subjects  were  required  to  awaken  at  0700  hours  (7:00  AM)  and  perfonn 
their  daily  activities  as  nonnal.  They  were  instructed  to  not  consume  any  caffeine  or  central  nervous 
system  (CNS)  altering  medications/substances  on  the  experimental  test  day.  Each  participant  arrived  at 
the  test  facility  at  2000  hours  (8:00  PM)  and  their  activity  data  was  analyzed  to  verify  that  proper  sleep 
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cycles  were  maintained.  Upon  verification,  the  subject  was  instrumented  with  the  EC6  monitoring 
equipment  and  EEG  sensors  to  record  brain  activity.  At  2100  hours  (9:00  PM),  subjects  began  the 
first  of  7  experimental  test  sessions.  They  completed  one  entire  battery  of  tasks  which  included  one 
session  of  the  target  acquisition  task  (30  minute  duration),  one  session  of  the  UAV  landing  task  (30 
minute  duration),  and  one  session  of  the  PVT  (10  minute  duration).  Additionally,  resting  EEG  data 
were  collected  with  the  subjects"  eyes  open  (2  minutes)  and  closed  (2  minutes)  both  before  and  after 
each  task.  This  provided  a  baseline  for  the  task  they  were  about  to  begin.  They  were  then  provided 
with  a  50-minute  break  of  free  time  during  which  they  could  participate  in  any  activity  except  for 
sleeping.  This  procedure  was  repeated  once  every  2  hours  for  7  repetitions.  The  experimental  test  day 
concluded  at  1100  hrs  (11 :00  AM)  the  following  morning.  Testing  started  at  14  hours  of  total  sleep 
deprivation,  and  continued  until  the  last  session,  which  started  at  26  hours  of  total  sleep  deprivation. 
After  the  last  session,  the  subject  was  brought  to  his/her  quarters  by  a  well-rested  member  of  the  staff. 


RESULTS  AND  DISCUSSION 


Performance  Data: 

Target  Acquisition  Task:  The  target  acquisition  data  files  provided  an  acquisition  time  for 
each  of  the  targets  in  each  trial.  These  acquisition  times  were  then  averaged  across  targets  and  trials 
for  each  of  the  seven  sessions  within  each  subject.  Next,  the  value  for  Session  1  was  denoted  as  the 
baseline  value  where  the  subject  was  rested  and  operating  normally.  The  remaining  averages  were 
normalized  across  subjects  by  calculating  the  acquisition  times  as  a  percentage  change  from  baseline. 
A  one-way  ANOVA  was  performed  using  these  normalized  values.  The  results  showed  a  significant 
main  effect  of  session  on  reaction  time  (F(6,63)=3.707,  p=003).  A  Bonferroni  post-hoc  test  (a=.05) 
was  used  to  examine  differences  between  the  seven  sessions.  The  results  found  a  significant  increase 
in  response  time  (p=. 023)  from  baseline  (Session  1)  to  Session  6  (Figure  1).  Increases  in  response  time 
indicate  an  increased  latency  and  therefore  a  degradation  in  perfonnance.  No  significant  results  were 
found  for  the  other  sessions.  The  average  percent  change  from  baseline  performance  was  28.131%. 

Simulated  UAV  Landing  Task:  The  UAV  landing  task  assessed  landing  performance  using 
glideslope  root  mean  square  error  (RMSE),  vertical  velocity  at  touchdown,  and  airspeed  at  touchdown. 
The  optimal  glideslope  of  4°  was  calculated  from  the  final  waypoint  to  the  touchdown  point  on  the 
runway.  RMSE  was  calculated  based  on  deviations  from  this  optimal  line.  A  one-way  ANOVA  was 
calculated  for  each  variable.  There  was  not  a  significant  effect  of  session  on  any  of  the  three 
perfonnance  metrics. 

Psychomotor  Vigilance  Task  (PVT):  Performance  on  the  PVT  was  assessed  via  three 
dependent  variables:  reaction  time,  lapses  (reaction  times  greater  than  5000  ms),  and  number  of  false 
starts.  A  one-way  ANOVA  was  performed  for  each  of  these  performance  metrics.  There  was  a 
significant  main  effect  for  session  on  the  reaction  times  (F( 6,  56)=3.652,  p=.004).  Specifically,  the 
Bonferroni  pairwise  comparison  (a=.05)  between  Session  1  and  Session  6  exhibited  a  significant 
difference  (p=.018).  The  mean  reaction  time  for  Session  1  was  239.8  msec  (se  =  14.2  msec)  whereas 
the  mean  for  Session  6  was  310.7  msec  (se  =  14.2  msec)  (Figure  2). 
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There  was  a  significant  difference  in  the  number  of  lapses  across  sessions  (F(6, 56)=3.672, 
p=.004).  Session  6  had  a  significantly  higher  number  of  lapses  than  both  Session  1  (p=.015)  and 
Session  3  (p=. 043)  (Figure  3)  according  to  the  Bonferroni  pairwise  comparisons  (a=.05).  No 
significant  differences  in  the  number  of  false  starts  were  detected  across  the  sessions. 
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Session  Number 


Figure  2.  Average  PVT  Response  Time 
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Figure  3 


Session  Number 


Figure  3.  Average  Number  of  Lapses  during  PVT 


EEG:  EEG  data  were  classified  into  the  four  standard  activity  bands  of  delta  (1.5-3  Hz),  theta 
(3.  0-8.0  Hz),  alpha  (8.0-13.0  Hz),  and  beta  (13.0-20.0  Hz)  by  calculating  the  power  spectrum 
(using  a  Hamming  window)  on  a  minimum  of  three  3 -second  epochs  within  each  EEG  segment  of 
interest.  This  procedure  was  used  for  both  the  EEG  collected  during  the  PVT  as  well  as  for  the  resting 
EEGs  collected  prior  to  and  immediately  following  execution  of  each  task.  Bands  were  analyzed 
separately  in  a  repeated  measures  AN OVA,  with  session  as  the  repeated  variable. 

For  the  target  acquisition  task,  there  was  a  significant  interaction  between  session,  eyes  open 
rest  condition,  and  the  eyes  closed  rest  condition  for  the  alpha  frequency  band  power  at  site  Cz.  (top, 
center  of  head)  (F(l,6)  =  2.70,  p  <  .05).  Cz  alpha  power  decreased  during  the  eyes  closed  condition  as 
the  level  of  sleep  deprivation  increased.  There  was  a  significant  main  effect  of  session  on  the  pre  and 
post  PVT  task  mean  alpha  power  at  site  Cz .  The  pre-PVT  alpha  power  was  4.273  (se  =  1.874)  while 
the  post  task  mean  was  3.700  (se  =  1 .729).  In  addition,  a  significant  main  effect  of  session  on  mean 
alpha  power  at  site  PZj  located  of  the  medial  parietal  cortex,  was  found  for  the  UAV  task  (F(6)  =  3.80, 
p<.05).  Alpha  power  declined  as  the  level  of  sleep  deprivation  increased. 

Shifts  in  delta  power  were  measured  for  the  target  acquisition  task  and  PVT.  There  was  a 
significant  main  effect  of  session  on  the  pre  and  post  target  acquisition  task  mean  delta  power  at  site 
C4 .  The  pre-task  mean  was  0.867  (se  =  0.266)  while  the  post  task  mean  was  1.028  (se  =0.224).  There 
was  a  significant  main  effect  of  session  on  the  pre  and  post  PVT  mean  delta  power  at  site  Cz .  The  pre¬ 
task  mean  was  1.346  (se  =0.238)  and  the  post-task  mean  was  1.747  (se  =  0.248).  There  was  a 
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significant  main  effect  of  session  on  the  mean  delta  power  at  site  Fz  during  the  PVT  task  F(  1,6)  = 
5.15,  p  <.01). 


The  EC6  eye-tracker  developed  by  EyeCom,  Inc.,  recorded  eye  pupil  position,  eye  state  (open 
or  closed),  and  pupil  size.  Eye  closure  data  were  extracted  from  the  final  data  files  and  used  to 
determine  the  total  amount  of  time  the  eyes  were  closed  during  each  task  as  a  proportion  of  the  total 
elapsed  time  to  complete  the  task.  The  proportion  function  can  be  found  in  Equation  1  where  PEC 
denotes  the  total  proportion  of  time  the  eyes  were  closed,  tc  is  the  total  duration  of  eye  closure,  and  ttotai 
refers  to  the  total  amount  of  time  to  complete  the  task: 


Equation  1.  Proportion  of  Session  Time  with  Eyes  Closed 

(  > 

PEC  = 


total 


To  normalize  the  data  across  subjects,  the  time  duration  data  were  calculated  as  a  percentage 
change  from  the  baseline  value  (Session  1).  The  equation  is  presented  below: 


Equation  2.  Percent  Change  from  Baseline 

%  Change  =  ^  -  t  A<  100 
t 


where  ?,•  is  the  total  eye  closure  time  of  the  ith  session  and  t  is  total  eye  closure  time  for  the  baseline 
session.  A  1-way  ANOVA  was  conducted  for  each  of  the  three  perfonnance  tasks  using  the  mean 
percent  change  in  eye  closure  time.  No  significant  main  effects  or  interactions  were  found  (p>0.05). 
Using  the  subject  means,  standard  2-tailed  t-tests  were  then  calculated  between  the  baseline  condition 
(Session  1)  and  the  remaining  six  sessions.  This  procedure  was  replicated  for  the  PVT  and  UAV 
landing  tasks.  The  results  from  data  gathered  during  the  target  acquisition  task  showed  mean 
differences  from  baseline  for  Sessions  3-7,  which  were  all  significantly  different  from  0  {p  =  0.329, 
0.104,  0.01 1,  0.013,  0.046  for  sessions  3,  4,  5,  6,  and  7,  respectively).  These  values  are  plotted  in 
Figure  4.  The  total  eye  closure  time  increased  with  the  level  of  sleep  deprivation  (over  3.5  times  the 
baseline  value  by  session  7).  Figure  5  illustrates  the  eye  closure  duration  percent  change  from  baseline 
for  the  PVT  task.  Mean  differences  from  baseline  for  sessions  3,  4,  6,  and  7  were  found  to  be 
significantly  greater  than  0 (p  =  0.038,  0.022,  0.043,  and  0.036,  respectively).  Additionally,  session  5 
approached  significance  ( p  =  0.063)  and  may  have  been  significant  with  a  greater  n-size.  Figure  6 
presents  the  eye  closure  duration  percent  change  from  baseline  values  collected  during  the  UAV 
landing  task.  Although  the  trend  appears  to  increase  with  higher  levels  of  sleep  deprivation,  the  mean 
differences  from  baseline  were  not  significantly  different  than  0. 

Approximate  Entropy:  EC6  eye  tracker  data  collected  during  the  target  acquisition  task  were 
analyzed  using  a  technique  known  as  approximate  entropy.  Approximate  entropy  (ApEn)  is  based  on 
the  simple  principle  that  if  a  time  series  signal  can  be  compared  with  itself  (heart  beat  data,  for 
example)  and  the  amount  of  the  disorder  in  a  comparison  or  change  (entropy)  between  relative  time 
shifts  of  these  data  is  increasing,  this  is  probably  an  indication  of  some  change  in  the  physiological 
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state.  This  differs  from  traditional  correlation  measures  since  they  are  not  based  on  infonnation  theory 
concepts  and  may  require  fixed  implicit  models.  The  use  of  ApEn  is  “model  independent”  and  only 
depends  on  the  real  time  data  series.  There  are  at  least  four  reasons  why  approximate  entropy 
provides  new  information  about  system  complexity  not  normally  derived  from  typical  statistical 
measures  (first  and  second  order  moments): 


(1)  If  data  are  noisy,  the  approximate  entropy  measure  can  be  compared  to  the  noise  level  in  the 
data  to  determine  what  quality  of  true  information  may  be  present  in  the  data. 

(2)  If  the  data  have  an  artifact,  this  does  not  impact  the  approximate  entropy  measure  as  much  as  it 
would  affect  typical  first  and  second  order  statistical  moments  from  the  data. 

(3)  Approximate  entropy  can  be  designed  to  work  for  small  data  samples  (n  <  50  points)  and  can 
be  applied  in  real  time,  on  line.  Thus,  changes  in  the  state  of  a  physical  process  may  be  quickly 
determined. 

(4)  For  pure  stochastic  processes,  approximate  entropy  will  become  practically  infinite. 


Thus,  the  quality  of  the  infonnation  in  a  signal  can  then  be  quantitatively  evaluated  by 
comparing  the  entropy  level  of  the  measured  signal  with  its  underlying  (non  random)  signal 
component.  The  statistical  analysis  of  the  pupil  position  ApEn  data  consisted  of  a  one-way  repeated 
measures  ANOVA.  The  independent  variable  was  session  number.  The  dependent  measure  was  the 
ApEn  values  and  the  data  were  considered  across  subjects. 
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Eyes  Closed  Time  Duration  as  a  Percent 
Change  from  Baseline 


1  2  3  4  5  6  7 

Session  Number 

Figure  4.  Target  Acquisition  Task.  Totay  Eye  Closure  Duration  Mean  Percent  change  from 

Baseline. 
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Eyes  Closed  Time  Duration  Percent  Change 
from  Baseline 


Figure  5.  PVT  Task.  Total  Eye  Closure  Duration  Mean  Percent  change  from  Baseline. 
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Eyes  Closed  Time  Duration  Percent  Change 
From  Baseline 


The  data  used  in  the  ApEn  analysis  were  the  root  mean  square  eye  position  about  a  center 
point.  To  perfonn  the  ApEn  analysis,  a  block  of  run  length  (m)  and  a  tolerance  window  (r)  must  be 
specified  to  compute  ApEn.  The  parameter  r  is  typically  20%  of  the  standard  deviation  in  the  data 
(Pincus  and  Kalman,  2004).  The  m=l  case  was  applied  to  the  fatigue  data,  which  simply  specifies  the 
determination  of  the  irregularity  of  the  data  between  the  series  s(t)  and  s(t+l)  and  with  s(t-l).  Since 
the  present  code  was  developed  for  10  consecutive  data  points  and  the  data  from  the  fatigue  study 
consisted  of  over  20,000  data  points  (33  Hz  sampling  rate  for  over  600  seconds),  a  Monte  Carlo 
approach  was  synthesized.  Each  of  the  runs  of  20,000  points  was  sampled  200  times  uniformly  and 
the  ApEn  measure  was  calculated  yielding  200  samples  of  ApEn  for  a  single  run.  The  mean  of  these 
sampled  values  as  well  as  the  maximum  ApEn  were  determined  from  these  200  samples.  Using  the 
sessions  2,  5  and  6,  Table  1  portrays  the  maximum  ApEn  from  the  200  samples  within  each  run  for  all 
ten  subjects. 

Figure  7  portrays  the  distribution  of  maximum  ApEn  values  across  sessions  2,  5  and  6.  It  is 
clear  that  as  the  session  number  increases  (going  into  fatigue),  there  is  lower  mean  ApEn  indicating 
less  irregularity  in  the  eye  movement  data.  A  Tukey-Kramer  test  was  performed  at  an  alpha  of  0.05 
indicating  that  Sessions  5  and  6  were  significantly  different  from  Session  2  but  Sessions  5  and  6  were 
not  statistically  different  each  other.  The  one-way  ANOVA  results  show  a  significant  effect  for 
session  (F(2,27)=l 0.364 1 ,  p=. 0005). 


Table  1,  Maximum  ApEn 

by  subject  number  and  session  number 


Subject  Number 

Session  2 

Session  5 

Session  6 

Subject  1 

0.965663 

0.802347 

0.321888 

Subject  2 

0.643775 

0.643775 

0.0602 

Subject  3 

0.965663 

0.643775 

0.643775 

Subject  4 

0.643775 

0.321888 

0.321888 

Subject  5 

0.940977 

0.643775 

0.321888 

Subject  6 

0.965663 

0.321888 

0.321888 

Subject  7 

0.940977 

0.643775 

0.802347 

Subject  8 

0.643775 

0.321888 

0.321888 

Subject  9 

0.802347 

0.643775 

0.643775 

Subject  10 

0.643775 

0.321888 

0.643775 
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Figure  7 


session 


Figure  7.  Maximum  ApEn  versus  Session  Number 


Discussion 

The  main  objective  of  this  experiment  was  to  evaluate  the  ability  of  eye  metrics  to  predict 
fatigue  related  performance  declines  in  relevant  air  operation  environments,  it  is  important  to  first 
verify  that  such  environmental  stressors  were  achieved.  Perhaps  the  most  well  established  method  of 
objectively  determining  the  onset  of  fatigue  is  through  EEG  analysis.  Typically,  the  EEG  of  rested, 
alert  individuals  is  comprised  mainly  of  beta  waves  (13-30  Hz),  although  alpha  activity  (8-12  Hz)  will 
dominate  when  the  individual  is  calm  or  has  eyes  closed.  Conversely,  theta  activity  (4-8  Hz)  becomes 
dominant  when  the  subject  is  entering  early  stages  of  sleep  and  delta  activity  (<4  Hz)  is  prevalent 
during  deep  sleep  (stages  3  and  4).  The  EEG  analyses  illustrate  that  alpha  activity  decreased  while  the 
delta  activity  increased  with  growing  levels  of  sleep  deprivation.  It  can  be  objectively  concluded  that 
participants  experienced  fatigue,  particularly  in  Sessions  5  and  6.  Perhaps  due  to  the  body"s  circadian 
rhythms  and  the  subjecf's  anticipation  of  the  conclusion  of  the  experimental  data  collection  day  (and 
therefore  impending  rest  cycle),  the  EEG  recovered  somewhat  for  Session  7  (0900-1010)  (Schmidt 
and  Collette,  2007). 

Objective  performance  from  the  fatigue  study  during  both  the  target  acquisition  and  PVT  tasks 
followed  the  EEG  frequency  content  shifts  with  significant  declines  in  Session  6  and  slight  recovery 
for  Session  7.  The  magnitude  of  the  performance  decrement  also  appears  to  depend  on  the  type  of  task 
perfonned.  Although  significant  declines  in  objective  performance  measures  were  found  in  both  the 
target  acquisition  task  and  the  PVT,  the  UAV  landing  task  performance  appeared  relatively  devoid  of 
any  fatigue  consequences.  It  is  likely  that  this  was  due,  in  part,  to  the  level  of  arousal  or  engagement 
produced  by  the  task.  Hebb  (1955)  originally  introduced  the  concept  of  the  influence  of  arousal  by 
defining  task  performance  as  a  normally  distributed  bell-shaped  curve  with  respect  to  arousal.  As  a 
result,  this  theory  produced  a  value  for  arousal  at  which  performance  is  optimal,  referred  to  as  the 
“optimal  level  of  arousal.”  Correspondingly,  the  theory  posits  that  tasks  with  extremely  low  or  high 
levels  of  engagement  will  result  in  degraded  performance.  Because  both  the  target  acquisition  and 
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PVT  tasks  were  relatively  simplistic  and  repetitive,  it  is  theorized  that  the  level  of  arousal  was  below 
the  optimal  level.  Due  to  the  relative  complexity  and  higher  difficulty  of  the  UAV  landing  task,  it  is 
highly  plausible  that  the  task  engendered  an  elevated  level  of  engagement/arousal  (closer  to  the 
“optimal  level  of  arousal”)  thereby  benefiting  perfonnance.  This  increased  perfonnance  may  have 
masked  any  negative  consequences  of  sleep  deprivation.  However,  it  should  be  noted  that  such  effects 
are  often  short-lived. 

Another  potential  factor  resides  in  the  fact  that  the  subjects  utilized  in  this  experiment  were  not 
pilots  and  did  not  have  any  previous  aircraft  piloting  experience.  Although  each  subject  was  trained  to 
the  point  that  his/her  performance  reached  a  plateau,  several  subjects  never  reached  ideal  proficiency. 

In  other  words,  their  performance  remained  consistently  poor  during  training.  Because  the  training 
criteria  was  to  reach  the  individuaf's  performance  plateau  and  not  to  reach  a  specific  level  of 
proficiency,  the  subjects  were  not  dismissed  from  participation  in  the  study.  During  data  collection, 
these  subjects  exhibited  large  variations  in  performance  that  may  have  also  contributed  to  a  masking  of 
fatigue  consequences. 

Significant  increases  in  the  proportion  of  eye  closure  time  metric  were  found  via  t-tests  hours 
prior  to  significant  changes  in  task  perfonnance  for  both  the  PVT  and  target  acquisition  tasks. 
Therefore,  the  results  support  the  notion  that  eye  metrics  can  be  utilized  to  predict  the  onset  of  fatigue 
before  the  negative  consequences  begin  to  manifest.  Combined  with  the  fact  that  the  cameras  are 
located  near  the  eye  (as  opposed  to  on  the  panel  or  dash)  thereby  reducing  the  likelihood  of  losing 
track  of  the  pupil/eye  closure,  the  results  suggest  that  the  technology  has  excellent  potential  in 
providing  fatigue  awareness  and  prediction  capability  in  Air  Force  environments.  Nevertheless,  it 
should  be  noted  that  the  current  EC6  technology  was  designed  for  laboratory  use  only  and  would  need 
to  be  integrated  into  the  pilot' 's  life  support  equipment  (e.g.  helmet  or  oxygen  mask)  to  be  useful  in  the 
operational  environment.  Likewise,  custom  algorithms  designed  to  monitor  eye  closure  duration  in 
near  real-time  would  need  to  be  designed  and  implemented. 

The  ApEn  analysis  illustrated  that  there  is  a  statistically  significant  reduction  in  the  relative 
disorder  of  the  pupil  tracking  signal  as  the  level  of  sleep  deprivation  increases.  This  supported  the 
hypothesis  that  as  increased  levels  of  fatigue  are  coupled  with  reductions  in  complexity  or  irregularity 
of  the  eye  movements  partially  as  a  result  of  an  increased  proportion  of  time  staring  at  the  screen 
rather  than  rapidly  searching  for  targets.  Lower  complexity  of  the  tracking  signal  translates  into  lower 
values  of  ApEn.  Also  the  use  of  the  ApEn  real  time  measure  may  be  a  predictor  of  an  increased 
fatigue  state.  Thus  a  negative  rate  of  change  of  ApEn  may  indicate  the  onset  of  fatigue.  The  time  rate 
of  change  of  ApEn  has  also  been  documented  to  be  an  excellent  predictor  in  other  settings  such  as  G- 
induced  loss  of  consciousness  (Repperger,  Albery,  and  Tripp,  2004;  Repperger,  Albery,  Tripp,  and 
McKinley,  2004)  and  as  a  leading  indicator  of  a  possible  change  in  financial  stock  market  data  (Pincus 
and  Kalman,  2004).  It  is  believed  that  the  ApEn  metric  will  be  particularly  useful  in  the  flight 
environment  due  to  the  fact  that  pilots  must  constantly  perfonn  instrument  crosschecks,  search  for 
targets,  etc.  which  require  continuous  eye  movements.  The  results  of  this  experiment  indicate  that 
these  movements  will  decline  in  complexity  as  fatigue  begins  to  set  in.  This  is  similar  to  findings  by 
Russo  et  al.  (1999)  that  indicated  decreases  in  saccadic  velocity  are  correlated  with  increased  sleep 
deprivation. 
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CONCLUSIONS  AND  FUTURE  RESEARCH 


The  results  of  the  experiment  provide  ample  evidence  that  the  eye  metrics,  such  as  the 
proportionate  total  time  of  eye  closure  and  approximate  entropy,  can  be  used  to  indicate  the  onset  of 
fatigue  in  advance  of  significant  changes  in  operator  performance. 

The  present  results  using  ApEn  indicate  this  metric  is  a  strong  leading  indicator  of  a  change  in 
state  (it  has  a  high  sensitivity).  Having  a  high  sensitivity,  however,  may  lead  to  many  false  positives 
and  the  use  of  ApEn  may  be  hampered  by  having  a  low  specificity.  Lack  of  specificity  in  this  case 
would  mean  the  ApEn  indicator  may  change  but  the  human  may  not  be  in  a  fatigue  state.  The  prior 
work  on  ApEn  has  shown  a  high  sensitivity  of  ApEn  as  a  leading  indicator,  but  additional  studies 
should  also  examine  the  specificity  aspects  of  using  ApEn.  However,  it  should  be  noted  that  the 
EyeCom,  Inc.,  technology  and  software  compliments  similar  attempts  to  monitor  operator  alertness 
such  as  those  found  in  Ji,  Zhiwei,  &  Lan  (2004).  In  fact,  it  is  suggested  that  the  metrics  presented  in 
this  study  should  be  employed  in  concert  with  additional  metrics,  such  as  saccadic  velocity  described 
by  Russo  et  al.  (2003)  and  increases  in  pupil  constriction  described  by  Russo  et  al.  (1999),  in  a  final 
system  development.  Such  additions  would  serve  to  reduce  the  risk  of  false  positive  errors. 

Finally,  because  the  cameras  were  mounted  on  eyeglass-like  frames,  the  system  was  able  to 
continuously  monitor  the  eye  throughout  all  sessions.  Overall,  the  system  consistently  and  reliably 
monitored  the  subjecf's  eye,  thereby  eliminating  field  of  view  constraints  characteristic  of  dash- 
mounted  systems.  It  should  be  noted  that  the  analyses  used  in  the  described  experiments  were 
performed  post  hoc  and  merely  provide  evidence  that  metrics  exist  that  can  be  monitored  in  real  time 
or  near  real  time  to  predict  negative  perfonnance  effects  from  stressors  such  as  fatigue.  Although  the 
EyeCom  device  does  collect  eye  state  data  in  real  time,  additional  algorithm  development  combined 
with  systems  integration  is  necessary  to  ensure  the  system  is  usable  in  the  flight/combat  environment. 
The  technology  remains  an  experimental  system  and  maintains  some  integration  and  comfort  issues. 
However,  the  current  design  was  not  intended  for  operational  use  and  will  need  to  be  customized  for 
specific  applications. 
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